Write metadata to a dedicated JSON file #737

marcelschurer · 2025-03-24T10:09:22Z

This PR adds the new function trixi2json, which exports simulation metadata to a dedicated JSON file at the start of the simulation. Previously, metadata was stored within the VTK output files. The structure and design of write_json.jl closely follow the existing implementation in write_vtk.jl for consistency.

This is what the JSON files looks like for the hydrostatic_water_comun_2d.jl example:

`boundary_1_metadata.json`:

{
  "state_equation": "StateEquationCole",
  "julia_version": "1.11.5",
  "boundary_model": "BoundaryModelDummyParticles",
  "density_calculator": "AdamiPressureExtrapolation",
  "viscosity_model": "Nothing",
  "solver_version": {
    "x": "v0.2.6-27-g7825eb968"
  },
  "smoothing_length": 0.06,
  "solver_name": "TrixiParticles.jl",
  "smoothing_kernel": "SchoenbergCubicSplineKernel"
}

`fluid_1_metadata.json`:

{
  "viscosity_alpha": 0.02,
  "smoothing_kernel": "SchoenbergCubicSplineKernel",
  "acceleration": [
    0.0,
    -9.81
  ],
  "julia_version": "1.11.5",
  "state_equation": "StateEquationCole",
  "density_calculator": "ContinuityDensity",
  "state_equation_rho0": 1000.0,
  "viscosity": "ArtificialViscosityMonaghan",
  "solver": "WCSPH",
  "viscosity_epsilon": 0.01,
  "state_equation_c": 10.0,
  "smoothing_length_factor": 0.8333333333333334,
  "correction_method": "Nothing",
  "state_equation_exponent": 7.0,
  "solver_version": {
    "x": "v0.2.6-27-g7825eb968"
  },
  "solver_name": "TrixiParticles.jl",
  "viscosity_beta": 0.0,
  "state_equation_pa": 0.0
}

Remark: Unit tests for this function will be added in a follow-up PR. In that upcoming PR, I will implement the functionality to read meta_data alongside the simulation data in read_vtk.jl.

… callbacks

codecov · 2025-05-19T09:22:38Z

Codecov Report

Attention: Patch coverage is 35.22013% with 103 lines in your changes missing coverage. Please review.

Project coverage is 69.65%. Comparing base (a91211d) to head (ff5651f).
Report is 5 commits behind head on dev.

Files with missing lines	Patch %	Lines
src/io/io.jl	0.00%	91 Missing ⚠️
src/io/write_vtk.jl	25.00%	6 Missing ⚠️
src/preprocessing/particle_packing/system.jl	0.00%	3 Missing ⚠️
src/callbacks/solution_saving.jl	0.00%	2 Missing ⚠️
src/callbacks/post_process.jl	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##              dev     #737      +/-   ##
==========================================
- Coverage   70.12%   69.65%   -0.48%     
==========================================
  Files         106      107       +1     
  Lines        6822     6913      +91     
==========================================
+ Hits         4784     4815      +31     
- Misses       2038     2098      +60

Flag	Coverage Δ
unit	`69.65% <35.22%> (-0.48%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

src/io/write_vtk.jl

src/preprocessing/particle_packing/system.jl

src/TrixiParticles.jl

src/io/io.jl

src/callbacks/post_process.jl

src/io/io.jl

LasNikas · 2025-06-04T09:08:46Z

/run-gpu-tests

efaulhaber · 2025-06-13T09:11:58Z

src/callbacks/solution_saving.jl

@@ -84,8 +81,8 @@ function SolutionSavingCallback(; interval::Integer=0, dt=0.0,
                                save_times=Float64[],
                                save_initial_solution=true, save_final_solution=true,
                                output_directory="out", append_timestamp=false,
-                                prefix="", verbose=false, write_meta_data=true,
-                                max_coordinates=Float64(2^15), custom_quantities...)
+                                prefix="", verbose=false, max_coordinates=Float64(2^15),


This is a breaking change!

efaulhaber · 2025-06-13T09:16:01Z

src/callbacks/post_process.jl

@@ -216,6 +216,8 @@ function initialize_postprocess_callback!(cb::PostprocessCallback, u, t, integra
    # Apply the callback
    cb(integrator)

+    write_meta_data(cb, integrator)


So this is called both by the postprocess and the savesolution callback? Do they both create the file and the second overwrites the first?

Do they both create the file and the second overwrites the first?

Yes

efaulhaber · 2025-06-13T09:20:03Z

src/io/io.jl

+    (; systems) = semi
+
+    output_directory = callback.output_directory
+    prefix = hasproperty(callback, :prefix) ? callback.prefix : ""


This is not clean. Why does the postprocess callback write this file anyway? This will mostly be used for restarting, no? Then it should be sufficient if only the saving callback writes this file.

One may start a simulation only with a postprocess callback without writing the solution.

src/io/io.jl

svchb · 2025-06-13T09:41:50Z

This is missing a test
The files need more structure

It should be one file for all the systems!

So something like this:

{
  "info":{
        "julia_version": "1.11.5",
        "solver_name": "TrixiParticles.jl",
        "solver_version:": "v0.2.6-27-g7825eb968"
   }
  "systems":[
     "fluid_1":{
        "phase": "fluid",
        "type": "WCSPH",
        "viscosity_model":{
           "type": "ArtificialViscosityMonaghan",
           "alpha": 0.02,
           "beta": 0.0,
           "epsilon": 0.01
          },
        "smoothing_kernel": "SchoenbergCubicSplineKernel",
        "state_equation":{
          "type": "StateEquationCole",
          "rho0": 1000.0,
          "c": 10.0,
          "exponent": 7.0,
          "pa": 0.0
        }
....

* add sgs * update * fix * remove * fix doc * fix * fixes * typo * tpo * format * add explanation * adjust tests * add to validation setup for wcsph * review fixes * Update taylor_green_vortex_2d.jl * Update validation_taylor_green_vortex_2d.jl * Update dam_break_2d.jl * fix and add citations * add news * implement suggestions * format * rename viscosity * too much renaming * implement suggestions * Update validation_taylor_green_vortex_2d.jl * fix tests again --------- Co-authored-by: Erik Faulhaber <44124897+efaulhaber@users.noreply.github.com> Co-authored-by: Niklas Neher <73897120+LasNikas@users.noreply.github.com>

marcelschurer added 3 commits March 24, 2025 08:39

Integrate write2json for initial metadata export

db89067

Remove old write_meta_data parameter from VTK writing functions and…

d8906ee

… callbacks

Add solver_name to metadata and format files

8558b13

marcelschurer changed the title ~~write2json~~ Write metadata to a dedicated JSON file Mar 24, 2025

marcelschurer added 4 commits April 14, 2025 11:45

Merge branch 'main' into write2json

86e491c

update write2json.jl to match merge

8803ef9

Merge branch 'main' into write2json

1ebda82

rename file

b86e2d6

marcelschurer added 5 commits May 19, 2025 11:22

update references

164b56e

rename function

2165d37

Refactor write2vtk!

51317e3

formatting

8f6eda9

refactor functions

7825eb9

LasNikas marked this pull request as ready for review May 21, 2025 08:47

LasNikas reviewed May 21, 2025

View reviewed changes

marcelschurer and others added 5 commits June 2, 2025 11:36

Merge branch 'trixi-framework:main' into write2json

e3cd5ca

implement suggestions

251eaf2

fix bugs

fecc01f

add add_meta_data! function for system::ParticlePackingSystem

3a399ff

relocate add_meta_data! to system.jl

9868c4a

LasNikas reviewed Jun 3, 2025

View reviewed changes

src/io/io.jl Outdated Show resolved Hide resolved

formatting

45940d9

marcelschurer requested a review from LasNikas June 4, 2025 07:38

LasNikas requested review from efaulhaber and svchb June 4, 2025 07:53

LasNikas approved these changes Jun 4, 2025

View reviewed changes

marcelschurer and others added 2 commits June 11, 2025 11:54

Merge branch 'main' into write2json

f6b605e

Merge branch 'main' into write2json

479e35c

efaulhaber requested changes Jun 13, 2025

View reviewed changes

svchb added breaking changes This change will break the public API and requires a new major release labels Jun 13, 2025

svchb modified the milestone: 0.4 Jun 13, 2025

efaulhaber changed the base branch from main to dev June 13, 2025 09:45

svchb and others added 4 commits June 13, 2025 13:26

Merge branch 'trixi-framework:main' into write2json

bb81f90

dispatch FluidSystem

7d34fe5

Write one unified metadata file for all systems

ff5651f

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Write metadata to a dedicated JSON file #737

Write metadata to a dedicated JSON file #737

Uh oh!

marcelschurer commented Mar 24, 2025 •

edited

Loading

Uh oh!

codecov bot commented May 19, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

LasNikas commented Jun 4, 2025

Uh oh!

efaulhaber Jun 13, 2025

Uh oh!

efaulhaber Jun 13, 2025

Uh oh!

LasNikas Jun 13, 2025 •

edited

Loading

Uh oh!

efaulhaber Jun 13, 2025

Uh oh!

LasNikas Jun 13, 2025

Uh oh!

Uh oh!

svchb commented Jun 13, 2025 •

edited by efaulhaber

Loading

Uh oh!

Uh oh!

Write metadata to a dedicated JSON file #737

Are you sure you want to change the base?

Write metadata to a dedicated JSON file #737

Uh oh!

Conversation

marcelschurer commented Mar 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

boundary_1_metadata.json:

fluid_1_metadata.json:

Uh oh!

codecov bot commented May 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

LasNikas commented Jun 4, 2025

Uh oh!

efaulhaber Jun 13, 2025

Choose a reason for hiding this comment

Uh oh!

efaulhaber Jun 13, 2025

Choose a reason for hiding this comment

Uh oh!

LasNikas Jun 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

efaulhaber Jun 13, 2025

Choose a reason for hiding this comment

Uh oh!

LasNikas Jun 13, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

svchb commented Jun 13, 2025 • edited by efaulhaber Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

marcelschurer commented Mar 24, 2025 •

edited

Loading

`boundary_1_metadata.json`:

`fluid_1_metadata.json`:

codecov bot commented May 19, 2025 •

edited

Loading

LasNikas Jun 13, 2025 •

edited

Loading

svchb commented Jun 13, 2025 •

edited by efaulhaber

Loading